This section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present invention that are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present invention. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
Recent advancements in display technology are beginning that allows an extended range of color, luminance and contrast to be displayed. Technologies allowing for extensions in luminance or brightness range of image content are known as high dynamic range imaging, often shortened to HDR. HDR images have at least one of the higher white level, higher contrast or higher bit depth than conventional (hereon referred to as LDR for low dynamic range) images. Today, a huge amount of legacy LDR content is available that need to be extended to HDR images, for example HDTV broadcast content or feature films color graded for classical cinema projection.
To prepare LDR content for HDR display devices, reverse or inverse tone mapping operators (iTMO) are employed. Such algorithms process the luminance level, contrast, hue and chroma in the image content with the aim of recovering or recreating the appearance of the original scene. Typically, iTMOs take a conventional (i.e. LDR) image as input, expand the luminance range of the colors of this image in a global manner, and subsequently process dark or bright regions locally to enhance the HDR appearance of colors in the image. Usually generated HDR images can be trusted, however, noise, clipping and quantization present in LDR images might be amplified when recreating HDR image. This is particularly problematic for images containing visual effects.
In today filmmaking but also gaming, visual effects (hereon referred to as VFX) cover the processes by which computer generated imagery is created and/or manipulated apart from the workflow of capturing images during a live action footage (hereon referred to as natural images). VFX, in non-limiting examples, comprise artificially created visual elements such as animated objects and characters, light effects, transparent objects such as fire, smoke and fog that are added to natural images in order to generate a final intended image (hereon referred to as image containing VFX). Some VFX are linked to light effects, for example specular reflections and cast shadows, and particularly induce dark or bright parts in the images. VFX are often created from physical models of light emission, reflection, transmission and absorption in a scene. These physical light models are part of so-called VFX assets. VFX assets contain all light, shape, texture and motion related information required for VFX generation. The physical light models use high numerical precision. From these models, images are rendered. This is the step where the high precision effects are baked into images of usually limited precision, for example due to quantization into 10 or 12 bits per color channel and pixel. According to another example related to gaming, images and synthetic objects are transmitted to a consumer device using MPEG-4 Binary Format for Scene (BIFS) in an audiovisual stream according to MPEG-4 Part 11. Then, the consumer device renders the synthetic objects and inserts them into the images. Again these kind of artificial images shows highlights, saturated colors and other light effects of high precision up to the quantization of the image signal. Since parts of LDR images related to VFX reveal significant highlights, dark shadows, saturated colors and generally light effects of high visual quality, the poor quality of VFX in HDR images that were generated by inverse tone mapping methods from LDR images containing VFX is particularly troublesome. Notably can be expected that parts of such HDR images suffer from clipping, luminance compression, lack of details, modification of saturation, noise amplification and banding.
Although several iTMO solutions exist such as different iTMO depending on parts of the image or iterative ITMO process, none of those methods particularly address the specificity of the images generated with VFX.